{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parallel Simulation for Parameter Exploration\n",
    "\n",
    "[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/brainpy/brainpy/blob/master/docs_version2/tutorial_simulation/parallel_for_parameter_exploration.ipynb)\n",
    "[![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/brainpy/brainpy/blob/master/docs_version2/tutorial_simulation/parallel_for_parameter_exploration.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "@[Tianqiu Zhang](mailto:tianqiuakita@gmail.com) @[Chaoming Wang](mailto:adaduo@outlook.com)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parameter exploration and selection is an essential part in brain dynamics modeling. In general, there are two problems for the parameter exploration:\n",
    "\n",
    "1. how to run multiple models concurrently?\n",
    "2. how to manage device memory allowing multiple models to run concurrently?\n",
    "\n",
    "First, most of the BrainPy models supports multiple kinds of parallelization, including parallelization of multi-threading and multi-processing on a single machine, and parallelization across multiple devices. In the below, we will illustrate these parallelization APIs one-by-one.\n",
    "\n",
    "Second, every call of a BrainPy model will consume a fraction of device memory. Therefore, BrainPy provides a API ``brainpy.math.clear_buffer_memory()`` for memory clean.\n",
    "\n",
    "In the following, we will illustrate how to combine them together to get an efficient parameter exploration for your models."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-10-06T04:04:30.500649Z",
     "start_time": "2025-10-06T04:04:26.277642Z"
    }
   },
   "source": [
    "import brainpy  as bp\n",
    "import brainpy .math as bm\n",
    "import numpy as np\n",
    "\n",
    "# bm.set_platform('cpu')"
   ],
   "outputs": [],
   "execution_count": 1
  },
  {
   "cell_type": "code",
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-10-06T04:04:30.513619Z",
     "start_time": "2025-10-06T04:04:30.506392Z"
    }
   },
   "source": [
    "bp.__version__"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.0.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parallelization across different CPU processors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parallelization across multiple CPU processors can be easily achieved with a single line of functional call ``brainpy.running.cpu_ordered_parallel()``. The following pseudocode demonstrates the usage of this API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "```python\n",
    "import brainpy as bp\n",
    "\n",
    "# define your function\n",
    "def run_model(par):\n",
    "  model = YourModel(par)\n",
    "  runner = bp.DSRunner(model)\n",
    "  runner.run(duration)\n",
    "  return runner.mon\n",
    "\n",
    "# define all parameter values need to explore\n",
    "all_params = [...]\n",
    "\n",
    "# run models in Jupyter\n",
    "results = bp.running.cpu_ordered_parallel(run_model, all_params, num_process=10)\n",
    "\n",
    "# run models in python file\n",
    "if __name__ == '__main__':\n",
    "  results = bp.running.cpu_ordered_parallel(run_model, all_params, num_process=10)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "We will use a simple HH neuron model as an example to show this kind of parallelization method. In this example, we use multi-processing technique to test four different current values as input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "First, define your running function with the well-defined `input` and `output` data."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-10-06T04:04:30.535340Z",
     "start_time": "2025-10-06T04:04:30.531782Z"
    }
   },
   "source": [
    "def hh_spike_num(bg_current): # \"input\" is the bg_current\n",
    "  import brainpy as bp  # needed to reimport packages when\n",
    "                        # run the function in Jupyter\n",
    "  model = bp.neurons.HH(1)\n",
    "  runner = bp.DSRunner(model, monitors=['spike'], inputs=['input', bg_current])\n",
    "  runner.run(1000.)\n",
    "  return runner.mon['spike'].sum()  # \"output\" is the spike number"
   ],
   "outputs": [],
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Then, define all your parameter spaces."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-10-06T04:04:30.585113Z",
     "start_time": "2025-10-06T04:04:30.550559Z"
    }
   },
   "source": [
    "current = bm.linspace(1, 10.1, 10)  # here only one parameter"
   ],
   "outputs": [],
   "execution_count": 4
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Finally, run your model concurrently with the  parallelization syntax."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-10-06T04:04:31.528718Z",
     "start_time": "2025-10-06T04:04:30.591995Z"
    }
   },
   "source": [
    "r = bp.running.cpu_ordered_parallel(hh_spike_num, [current], num_process=10)\n",
    "\n",
    "r"
   ],
   "outputs": [
    {
     "ename": "NotImplementedError",
     "evalue": "Multiprocessing is not available in Python >=3.11 on Windows. Please use Linux or MacOS, or Windows with Python <= 3.10.",
     "output_type": "error",
     "traceback": [
      "\u001B[31m---------------------------------------------------------------------------\u001B[39m",
      "\u001B[31mNotImplementedError\u001B[39m                       Traceback (most recent call last)",
      "\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[5]\u001B[39m\u001B[32m, line 1\u001B[39m\n\u001B[32m----> \u001B[39m\u001B[32m1\u001B[39m r = bp.running.cpu_ordered_parallel(hh_spike_num, [current], num_process=\u001B[32m10\u001B[39m)\n\u001B[32m      3\u001B[39m r\n",
      "\u001B[36mFile \u001B[39m\u001B[32mD:\\codes\\projects\\BrainPy\\brainpy\\version2\\running\\pathos_multiprocessing.py:177\u001B[39m, in \u001B[36mcpu_ordered_parallel\u001B[39m\u001B[34m(func, arguments, num_process, num_task, **tqdm_kwargs)\u001B[39m\n\u001B[32m    129\u001B[39m \u001B[38;5;250m\u001B[39m\u001B[33;03m\"\"\"Performs a parallel ordered map with a progress bar.\u001B[39;00m\n\u001B[32m    130\u001B[39m \n\u001B[32m    131\u001B[39m \u001B[33;03mExamples::\u001B[39;00m\n\u001B[32m   (...)\u001B[39m\u001B[32m    169\u001B[39m \u001B[33;03m  A list which will apply the function to each element of the given tasks.\u001B[39;00m\n\u001B[32m    170\u001B[39m \u001B[33;03m\"\"\"\u001B[39;00m\n\u001B[32m    171\u001B[39m generator = _parallel(\u001B[38;5;28;01mTrue\u001B[39;00m,\n\u001B[32m    172\u001B[39m                       func,\n\u001B[32m    173\u001B[39m                       arguments,\n\u001B[32m    174\u001B[39m                       num_process=num_process,\n\u001B[32m    175\u001B[39m                       num_task=num_task,\n\u001B[32m    176\u001B[39m                       **tqdm_kwargs)\n\u001B[32m--> \u001B[39m\u001B[32m177\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mlist\u001B[39m(generator)\n",
      "\u001B[36mFile \u001B[39m\u001B[32mD:\\codes\\projects\\BrainPy\\brainpy\\version2\\running\\pathos_multiprocessing.py:71\u001B[39m, in \u001B[36m_parallel\u001B[39m\u001B[34m(ordered, function, arguments, num_process, num_task, **tqdm_kwargs)\u001B[39m\n\u001B[32m     45\u001B[39m \u001B[38;5;250m\u001B[39m\u001B[33;03m\"\"\"Perform a parallel map with a progress bar.\u001B[39;00m\n\u001B[32m     46\u001B[39m \n\u001B[32m     47\u001B[39m \u001B[33;03mParameters::\u001B[39;00m\n\u001B[32m   (...)\u001B[39m\u001B[32m     68\u001B[39m \u001B[33;03m    in parallel in order with a progress bar.\u001B[39;00m\n\u001B[32m     69\u001B[39m \u001B[33;03m\"\"\"\u001B[39;00m\n\u001B[32m     70\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m sys.platform == \u001B[33m'\u001B[39m\u001B[33mwin32\u001B[39m\u001B[33m'\u001B[39m \u001B[38;5;129;01mand\u001B[39;00m sys.version_info.minor >= \u001B[32m11\u001B[39m:\n\u001B[32m---> \u001B[39m\u001B[32m71\u001B[39m     \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mNotImplementedError\u001B[39;00m(\u001B[33m'\u001B[39m\u001B[33mMultiprocessing is not available in Python >=3.11 on Windows. \u001B[39m\u001B[33m'\u001B[39m\n\u001B[32m     72\u001B[39m                               \u001B[33m'\u001B[39m\u001B[33mPlease use Linux or MacOS, or Windows with Python <= 3.10.\u001B[39m\u001B[33m'\u001B[39m)\n\u001B[32m     74\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m ProcessPool \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m \u001B[38;5;129;01mor\u001B[39;00m cpu_count \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[32m     75\u001B[39m     \u001B[38;5;28;01mraise\u001B[39;00m PackageMissingError(\n\u001B[32m     76\u001B[39m \u001B[38;5;250m        \u001B[39m\u001B[33;03m'''\u001B[39;00m\n\u001B[32m     77\u001B[39m \u001B[33;03m      Please install \"pathos\" package first. \u001B[39;00m\n\u001B[32m   (...)\u001B[39m\u001B[32m     80\u001B[39m \u001B[33;03m        '''\u001B[39;00m\n\u001B[32m     81\u001B[39m     )\n",
      "\u001B[31mNotImplementedError\u001B[39m: Multiprocessing is not available in Python >=3.11 on Windows. Please use Linux or MacOS, or Windows with Python <= 3.10."
     ]
    }
   ],
   "execution_count": 5
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "However, the above usage will accumulate buffer memory in the running device. If your single model occupies too much memory, the out-of-memory error will be raised during the parameter exploration.\n",
    "\n",
    "A simple way to solve this issue is clear all buffers after each running of the function. For example, before returning your results, call `brainpy.math.clear_buffer_memory()` first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def hh_spike_num2(bg_current): # \"input\" is the bg_current\n",
    "  import brainpy as bp  # needed to reimport packages when\n",
    "                        # run the function in Jupyter\n",
    "\n",
    "  bg_current = bp.math.as_jax(bg_current)\n",
    "  model = bp.neurons.HH(1)\n",
    "  runner = bp.DSRunner(model, monitors=['spike'], inputs=['input', bg_current])\n",
    "  runner.run(1000.)\n",
    "\n",
    "  bp.math.clear_buffer_memory()\n",
    "  return runner.mon['spike'].sum()  # \"output\" is the spike number"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Note that ``clear_buffer_memory()`` will clear all JAX arrays in the device, therefore, it's better to give inputs as NumPy arrays, and return outputs as NumPy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a5c89fed0063425a8e533ac9ede63ccb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[0, 0, 1, 0, 0, 57, 60, 58, 65, 68]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "current = np.linspace(1., 10., 10)\n",
    "\n",
    "r = bp.running.cpu_ordered_parallel(hh_spike_num2, [current], num_process=10)\n",
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "If you think that the order of the running results does not matter, you can also use ``cpu_unordered_parallel()`` function. This can maximize the running efficiency of all processors, since all workers run with a non-blocking and unordered manner."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Parallelization with `jax.vmap`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "The second approach of realizing multi-threading parallelization is the vectorization map of JAX `jax.vmap`. `jax.vmap` vectorizes functions by compiling the mapped axis as primitive operations. It can avoid the recompilation of models in the same batch, and automatically parallelize the model running on the given machine. Following pseudocode demonstrates how simple of this parallelization approach is."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "```python\n",
    "from jax import vmap\n",
    "\n",
    "def run_model(par):\n",
    "  model = YourModel(par)\n",
    "  runner = bp.DSRunner(model)\n",
    "  runner.run(duration)\n",
    "  return runner.mon\n",
    "\n",
    "# define all parameter values need to explore\n",
    "all_params = [...]\n",
    "\n",
    "# batch simulation through jax.vmap\n",
    "r = vmap(run_model)(*all_params)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Note that if you have too many parameters to search, ``jax.vmap`` will consume too much memory. For this time, you can use our wrapped API ``brainpy.running.jax_vectorize_map()``, which controls the running batch size by ``num_parallel`` parameter. You can set a smaller value of ``num_parallel`` when your device memory is not enough (no matter on the CPU or GPU device)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def hh_spike_num3(bg_current): # \"input\" is the bg_current\n",
    "  model = bp.neurons.HH(1)\n",
    "  runner = bp.DSRunner(model, monitors=['spike'], inputs=['input', bg_current],\n",
    "                           numpy_mon_after_run=False)\n",
    "  runner.run(1000.)\n",
    "  return runner.mon['spike'].sum()  # \"output\" is the spike number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "58c9276118ed47ed84b3fd4ab0491d0d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d7c9093f42ab428eb8bad159b1631b41",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7428ef7ff11d4f7dbd233db31b8dce48",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5419cb9f45bf4b2aa01cff6cc6211657",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Array([ 0,  0,  0,  0,  0, 45, 60, 63, 66, 68], dtype=int32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "current = bm.linspace(1., 10.1, 10)\n",
    "r = bp.running.jax_vectorize_map(hh_spike_num3, [current], num_parallel=3)\n",
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "The function throw into the ``jax_vectorize_map()`` can not call ``clear_buffer_memory()``. Otherwise will raise errors. Instead, uses can set ``clear_buffer=True/False`` using ``jax_vectorize_map()``. For such kind of usage, all inputs and outputs will be automatically transformed in to NumPy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7c795b93d5f54c04afd87d98c8ef96a9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "55c1901f49e64103ae02cb8cce5f84c3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "afa22c9545634deb9e260b33e51042c9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3e2d3a85188e45e29ce22d1d42fb853f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  1,  0,  0, 57, 60, 63, 66, 68])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "current = bm.linspace(1., 10.1, 10)\n",
    "r = bp.running.jax_vectorize_map(hh_spike_num3, [current], num_parallel=3, clear_buffer=True)\n",
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Parallelization across multiple devices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "BrainPy support parallelization running on multiple devices (e.g., multiple GPU devices or TPU cores) or HPC systems (e.g., supercomputers). Different from the above thread-based and processor-based parallelization methods, in which the same model runs in parallel on the same device, device-based parallelization runs the same model in parallel on multiple devices.\n",
    "\n",
    "One way to express the multi-device parallelization of BrainPy models is using `jax.pmap` instruction. JAX delivers `jax.pmap` to express SIMD programs. It provides an interface to run the same model on multiple devices with different parameter values. It usage is analogy to `jax.vmap`. Following pseudocode presents an example to run BrainPy models on multiple devices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "```python\n",
    "from jax import pmap\n",
    "\n",
    "def run_model(par):\n",
    "  model = YourModel(par)\n",
    "  runner = bp.DSRunner(model)\n",
    "  runner.run(<int>)\n",
    "  return runner.mon\n",
    "\n",
    "# define all parameter values need to explore\n",
    "all_params = [...]\n",
    "\n",
    "# parallel simulation through jax.pmap\n",
    "r = pmap(run_model)(*all_params)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "``jax.pmap`` has the similar issue to ``jax.vmap`` when you parallelize across many parameters. This time you can use the wrapped function ``brainpy.running.jax_parallelize_map()``.\n",
    "\n",
    "If you are using ``pmap`` in you CPU device, you can set the virtual number of the device by calling ``brainpy.math.set_host_device_count(n)``. Then, you can call ``jax_parallelize_map()`` safely one your CPU platform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "866deb0b6d91493e84d26232c9b53045",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "array([ 0,  0,  0,  0,  0,  0,  0, 49, 52, 54, 56, 58, 59, 61, 62, 63, 65,\n",
       "       66, 67, 68])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bp.math.set_host_device_count(10)  # this should place on the top of the file\n",
    "\n",
    "current = bm.linspace(1., 10.1, 20)\n",
    "r = bp.running.jax_parallelize_map(hh_spike_num3, [current], num_parallel=10, clear_buffer=True)\n",
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "BrainPy also works well with job scheduling systems such as SLURM on a supercomputer center. Therefore, another way to express multi-device parallelization is to employ the classical resource management system. Following script demonstrates an example that submits a batch script to SLURM."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "```python\n",
    "\n",
    "#!/bin/bash\n",
    "#SBATCH -J <name>\n",
    "#SBATCH -o <file name>\n",
    "#SBATCH -p <str>\n",
    "#SBATCH -n <int>\n",
    "#SBATCH -N <int>\n",
    "#SBATCH -c <int>\n",
    "\n",
    "python your_script.py\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "f37317bd3e2379aba54e3aa76414bc918141342cb86849b10e642bf3607e7693"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
